(19) 



J 




■ill 



(12) 



(43) Date of publication: 

30.06.1999 Bulletin 1999/26 

(21) Application number: 98310040.5 

(22) Date of filing: 08.12.1998 



Europaisches Patentamt 
European Patent Office 
Office europeen des brevets (11) EP 0 926 678 A2 

EUROPEAN PATENT APPLICATION 

(51) IntCI. 6 : G11B 27/031 



(84) Designated Contracting States: 


(72) Inventors: 


ATBECHCYDEDKESR FRGBGRIEITU LU 


• Cen, Shanwei 


MCNLPTSE 


Aloha, Oregon 97006 (US) 


Designated Extension States: 


• Brunner, Theodore O. 


ALLTLVMKROSI 


Portland, Oregon 9721 0 (US) 


(30) Priority: 17.12.1997 US 992684 


(74) Representative: 


(71) Applicant: TEKTRONIX, INC. 


Molyneaux, Martyn William et al 


Langner Parry 


Wilsonville, Oregon 97070-1000 (US) 


52-54 High Holborn 




London WC1V6RR(GB) 



CM 
< 

GO 
1^ 
CO 

CO 
CM 

o> 



(54) Method and apparatus for compressed video segment cut-and-concatenation for editing 



(57) A method and apparatus for compressed video 
segment cut and concatenation for editing uses a re- 
encoder (9), controller (1 1). ancillary storage (15), and a 
single decoder (13). Video segments from video dips 
stored on a server are chosen for playback in a specific 
order. References to the frames of the chosen 
sequence are stored by use of index pointers in ancil- 



lary storage and P or B frames near the cut points 
between segments are re-encoded to I frames for future 
insertion and re-ordered in storage for forward or 
reverse playback of the selected sequence by a single 
decoder. 
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Description 

[0001 ] The present invention relates in general to dig- 
ital video, and more particularly to motion-prediction 
compressed digital video segment cut and concatena- 5 
tion for editing. 

[0002] An example of a compression scheme using 
motion precfiction is the Moving Picture Expert Group 
(MPEG) compression standard. MPEG (ISO/IEC 
11172-2 and 13818-2) uses motion prediction tech- 10 
niques to reach a high compression ratio. A com- 
pressed MPEG video stream consists of a sequence of 
I, P t and B frames. An independent (I) frame is self con- 
tained, containing no motion prediction, while a predic- 
tive (P or B) frame is a difference between the current 15 
frame and future or past reference frame(s). Due to 
motion prediction, MPEG video has the advantage of a 
lower bandwidth (bit-rate) requirement than other com- 
pression schemes such as the Joint Photographic 
Expert Group (JPEG) standard. On the other hand, 20 
motion prediction makes MPEG video more compli- 
cated during editing. MPEG video does not support 
editing well, due to difficulties with variable speed, for- 
ward and backward play; frame-accurate random 
access, and frame-accurate cut-in and cut-out. 25 
[0003] Two components can be identified in a video 
editing system: a video media storage server and an 
editor. The media storage server contains source video 
dips, which are immutable during editing. The editor is 
used to browse video dips from the media storage 30 
server, identify segments from the source dps, and 
construct edit-dedsion-lists (EDUs) referencing the 
identified segments. It is necessary for the editor to be 
able to play EDL's as composite video dips, so that the 
effect of the EDL's can be viewed. The editor needs to 35 
smoothly play across the cut-in and cut-out points, 
backward and forward. 

[0004] When attempting to play across the cut points, 
it is not sufficient to simply cut the segments out of the 
source video dips, on frame boundaries, and concate- 40 
nate the segments together. Some P or B frames close 
to th cut-in and cut-out points, which are necessary for 
play, are non-decodable, since their reference frames 
are missing from the segment 

[0005] A current method of re-encoding P and B 45 
frames dose to the cut point exists, but it is expensive 
because it requires two decoders, and is limited 
because it does not provide for reverse playback. 
[0006] What is desired is an inexpensive motion-pre- 
diction compressed video editing system that enables so 
the editor to smoothly play forward and reverse across 
the cut-in and cut-out points that does not require multi- 
ple decoders. 

Summary of the Invention 55 

[0007] In accordance with the illustrated preferred 
embodiment of the invention, a method and apparatus 



is disdosed to cut motion-prediction compressed 
source video dips into segments, and concatenate 
them for smooth playback across cut-in and cut-out 
points. Whenever a video segment is identified, the oth- 
erwise non-decodable P and B frames within it at cut-in 
and cut-out points are decoded and re-encoded into I 
frames. The re-encoded I frames as well as other data 
structures which associate the original and re-encoded 
frames and identify frames within the segments are 
stored in andllary storage inside the editor. During play- 
back of an EDL, the data structures are consulted to 
determine which frames to play, and whether the origi- 
nal frames or the re-encoded frames should be used. 
Smooth playback of EDL's across the cut-in and cut-out 
points is provided, while using a single MPEG decoder 
and storing the source video dips in a media storage 
server unchanged. 

[0008] The objects, advantages, and other novel fea- 
tures of the present invention are apparent from the fol- 
lowing detailed description when read in conjunction 
with the appended daims and attached drawing. 

Brief Description of the Drawing 

[0009] 

Fig. 1 is a block diagrammatic view of a video edit- 
ing system according to the present invention. 

Fig. 2 is a block diagrammatic view of a motion-pre- 
diction compressed video editor according to the 
present invention. 

Fig. 3 rs a table showing examples of how frames 
are re-encoded near an editing cut point according 
to the present invention. 

Pesprfption of the Preferred E mbodiment 

[0010] Referring to Fig. 1, a motion-precfiction com- 
pressed video editing system has a video media stor- 
age device 5 for storage and output of various source 
video dps coupled to an editor 7. The editor cuts and 
concatenates segments from the source video dips and 
then outputs the resultant video stream in forward or 
reverse directions and at variable speed without chang- 
ing the source video resident on the video media stor- 
age server 5. 

[001 1 ] Referring to Fig. 2, a more detailed diagram of 
the editor 7 of Fig. 1 is shown. As video from the video 
media storage server 5 is input to a controller 11, the 
segments chosen are identified and referenced on an 
edit decision list (EDL) to be played by the editor as a 
composite video dip. The EDL, as well as index pointers 
to the frames in the segments identified on the EDL, are 
stored in andllary storage 15 which is coupled to the 
controller 11 . The source material resident in the video 
media storage server 5 remains unchanged. 
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[001 2] As each segment chosen is identified, the oth- 
erwise non-decodable P and B frames within it at the 
b ginning and end of the identified segment are 
d coded by a de-coder 13 and re-encoded, using infor- 
mation from the adjacent frames, into I frames by a re- 5 
encoder 9. The decoder 13 and reencoder 9 are both 
coupled to the controller 11 and ancillary storage 15. 
The re-encoded I frames, which can now be independ- 
ently decoded are placed in the ancillary storage 15. 
The association between the re-encoded I frames and 10 
their original version is stored in the ancillary storage 15 
for future reference. These re-encoded I frames, and if 
necessary, some existing I frames adjacent and close to 
the re-encoded I frames are then reordered. This is 
done by updating the index pointers to the frames in the 15 
segments stored in ancillary storage 15. In this way, the 
segments are made self-contained for playback in both 
directions and at variable speed. Also, the frames at the 
end and beginning of each identified segment (cut 
points) can be crossed backward and forward smoothly. 20 
The process for reverse playback is addressed and dis- 
closed further in patent application 6207 US. 

[0013] Which P and B frames to re-encode around a 
cut point and which frames surrounding them to re- 
order depend on the types of frames around the cut 25 
points. To help explain how the re-encoded frames and 
those surrounding them are re-ordered according to the 
present invention, several examples of P and B frame 
re-encoding and the re-ordering of frames surrounding 
them are given in Fig. 3. The display order and bit- 30 
stream order of an example video clip that is present on 
th video media storage server 5 is given in Fig. 3. Cut- 
ting the dip in the middle results in two identified seg- 
ments. The first with a cut-out point and the second 
with a cut-in point. In the three typical cases of cutting 35 
after 13, B4, and B5 (in display order), and after re- 
encoding of the otherwise non-decodable B frames, the 
result is the order of frames listed in the table shown in 
Fig. 3. The segment frames in the table are shown in bit- 
stream order. The frames in 0 represent re-encoded I 40 
frames and the frames in 0 represent the re-ordered 
frames. 

[0014] The video segments thus far identified on the 
EDL are then played back. By use of the controller 11, 
the editor examines the index pointers stored in andl- 45 
lary storage 15 corresponding to the video segments 
identified on the EDL and decodes those frames 
indexed by use of the decoder 13 for display. If a frame 
is found to have been stored in andllary storage 15 as a 
re-encoded I frame, that re-encoded version is then so 
decoded by decoder 13, and displayed instead of the 
original one. 

[0015] Playback of an EDL can either be h Id back 
until all the otherwise non-decodable P and B frames 
are re-encoded, or can happen in parallel with the re- 55 
encoding process, as long as the unfinished cut points 
are not played. This is effected by the controller 11. The 
hekJ-back method of the preferred embodiment how- 



ever, imposes no meaningful restriction on the respon- 
siveness of the editor 7. The inherent operational delay 
of the operator needing time to move a mouse and/or 
press buttons after each segment is identified is enough 
to allow the re-encocfing process to complete. This is 
because the re-encoding process is simple and quick 
since at each cut point only a few P and B frames need 
to be re-encoded into I frames and no time consuming 
motion prediction is involved. 
[0016] Thus the present invention provides an inex- 
pensive motion-prediction compressed video editing 
system that enables the editor to smoothly play forward 
and reverse across the cut-in and cut-out points that 
does not require multiple decoders. Additions, dele- 
tions, and other modifications of the disdosed particular 
embocfiments of the invention will be apparent to those 
in the art and are within the scope of the following 
daims. 

Claims 

1. A method for compressed video segment cut and 
concatenation for editing comprising the steps of: 

choosing video segments from stored video 
dips for composite playback in specific order; 

storing references to frames of the chosen 
video segments; 

re-encocfing compressed predictive frames at 
cut points between the chosen video segments 
to decodable compressed independent frames; 

decoding sequentially for playback the frames 
of the chosen segments according to the 
stored frame references by using a single 
decoder; 

substituting the re-encoded compressed inde- 
pendent frames for the compressed predictive 
frames as each cut point between the chosen 
segments is encountered during the sequential 
decoding. 

2. The method according to daim 1 , further compris- 
ing the step of updating the stored references to the 
frames of the chosen segments to effect a re-order- 
ing of the re-encoded compressed independent 
frames and other frames dose to the cut points 
between the chosen segments such that the cho- 
sen segments are made self contained for forward 
and revers playback. 

3. The method according to claim 1 or 2 further com- 
prising the step of starting the decoding step before 
all the compressed predictive frames at cut points 
between all the chosen segments are re-encoded 
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into compressed independent frames. 

4. An apparatus for compressed video segment cut 
and concatenation for editing comprising: 

5 

means for choosing video segments from 
stored video clips for composite playback in 
specific order; 

means for storing references to frames of the io 
chosen video segments; 

means for re-encoding compressed predictive 
frames at cut points between the chosen seg- 
ments to decodabie compressed independent is 
frames; 

a single decoder for decoding sequentially for 
playback the frames of the chosen segments 
according to the stored frame references; 20 

means for substituting the re-encoded com- 
pressed independent frames for the com- 
pressed independent frames as each cut point 
between the chosen segments is encountered 25 
during the sequential decoding. 

5- The apparatus of claim 4, further comprising means 
for updating the stored references to the frames of 
the chosen segments to effect a re-ordering of the 30 
re-encoded compressed independent frames and 
other frames close to the cut points between the 
chosen segments such that the chosen segments 
are made self contained for forward and reverse 
playback. 35 

6. The apparatus of claim 4 or 5 further comprising 
means for starting the decocting step before ail the 
compressed predictive frames at cut points 
between all the chosen segments are re-encoded 40 
into compressed independent frames. 

7. A compressed video editing system comprising: 

a video media storage server; 45 

a controller coupled to the server, operative for 
reading video clips from the video media stor- 
age server, choosing video segments from the 
video clips for playback, re-ordering frames so 
within those video segments to enable play- 
back, and signaling decode, encode and stor- 
age commands with respect to those video 
segments; 

55 

a re-encoder coupled to the controller, opera- 
tive for re-encoding compressed predictive 
frames into compressed independent frames at 



chosen cut points; 

ancillary storage coupled to the re-encoder and 
controller, operative for storing the re-encoded 
compressed independent frames and refer- 
ences to chosen video segment frames for 
playback; 

a single decoder coupled to the ancillary stor- 
age and controller, operative for decoding and 
playing back prediction-compressed video 
streams in real time according to the stored ref- 
erences to the chosen video segment frames 
and re-encoded compressed independent 
frames. 
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DISPLAY ORDER: 10 B1 B2 13 B4 B5 16 B7 B8 19 
BITSTREAM ORDER: 10 13 B1 B2 16 B4 B5 19 B7 B8 
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SEGMENT 1 


SEGMENT 2 


AFTER 13 
AFTER B4 
AFTER B5 


10 13 B1 B2 

10 13 B1 B2 ( [14] ) 

10 13 B1 B2 ( [14 15] ) 


( CI4 15] 16) 19 B7 B8 
( [15] 16) 19 B7 B8 
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